Clean Version Of The Pending Claims Under 37 C,F,R. $ 1.121(c)(3): 

Claims 1-42 now pending, are submitted below in accordance with 37 
C.F.R. §1.12 1(c)(3), which presents a clean version of the entire set of pending 
claims. . 



1. (Once Amended) A method comprisir 

identifying, in response to a search query/first multimedia objeefs having 
an associated keyword that matches a keyword in the search qu^ and second 
multimedia objects that have content features similar to/mose of the first 
multimedia objects; 

presenting the first and second multimedia objects to a user; 

monitoring feedback from the user as toywhich of the first and second 
multimedia objects are relevant to the search query; and 

annotating one or more of the njultimedia objects, which are deemed 
relevant by the user, with the keyword. 

2. A method as recited in claim 1, further comprising: 

maintaining associations/^tween the keywords and the multimedia objects, 
the associations being weigljted to indicate how relevant the keywords are to the 
multimedia objects; and 

adjusting the wei^ts of the associations based on the user's feedback. 

3. A memod as recited in claim 2, wherein the adjusting comprises 
increasing a weight of an association between the keyword and a particular 
multimedia obj^t that is deemed relevant by the user. 
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- 4. A method as recited in claim 2, wherein the adjusting cortipnses 
decreasing a weight of an association between the keyword and ^particular 
multimedia object that is deemed irrelevant by the user. 

5, A method as recited in claim 4, further comprising removing the 
keyword from the particular multimedia object in an event ^mat the weight is less 
than a threshold value. 



6- A method as recited in claim 1 , further comprising training how the 
first and second multimedia objects are identifiea based on the user's feedback. 




7. A method as recited in claim 1, further comprising refining the 



search to identify additional multiniedia objects that contain content features 



similar to those of the multimedia objects indicated by the user as being relevant. 




8. A method as recited in claim 1, wherein the multimedia objects 
comprise one of digital imafges, video objects, and audio objects. 



9* A computer readable medium having computer-executable 
instructions that, v^en executed on a processor, perform the method as recited in 
claim 1. 



10. /A method comprising: 
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iteratively retrieving multimedia objects from a database and monitoring 
feedback from a user as to whether the muhimedia objects are relev^it to a 
keyword in a search query; and / 

annotating the multimedia objects based on the user's feejioack, with the 
keyword. / 

11. A method as recited in claim 10, whereir/the retrieving comprises 
using content-based information retrieval to retrieve rtfe multimedia objects. 

12. A method as recited in claim 10; wherein the retrieving comprises 
using both content-based information retrieval and semantic-based information 
retrieval to retrieve the multimedia objeats. 

13. A method as recited /iii claim 10, wherein the monitoring comprises 
monitoring both feature-based relevance feedback and semantic-based relevance 
feedback. / 

14. A methodyQs recited in claim 10, wherein the annotating is hidden 
from the user. / 

15. A niethod as recited in claim 10, wherein the annotating comprises: 
in an evOTt that a particular multimedia object is deemed relevant by the 

user and is not yet annotated with the keyword, adding the kejovord to the 
particular multimedia object; and 

/ 
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in an event that the particular multimedia object is deemed relevant hy^me 
user and is already annotated with the keyword, strengthening an ass9Ciation 
between the keyword and the particular multimedia object, 

16. A method as recited in claim 10, wherein the annotating comprises: 
in an event that a particular multimedia object is deeiired irrelevant by the 

user and is already annotated with the keyword, weakening/^n association between 
the keyword and the particular multimedia object. 

17. A computer readable mediurt/ having computer-executable 
instructions that, when executed on a processor, perform the method as recited in 
claim 10. 



18. A method comprising: 

retrieving multimedia obje/ts according to a content-based retrieval 
process; 

presenting the multimedik objects to a user; 

monitoring feedback yftom the user as to which of the multimedia objects 
are relevant; and 

annotating one pr more of the multimedia objects based on the user's 
feedback, with a ke^ 




19. A/fnethod as recited in claim 18, wherein the monitoring comprises 
monitoring b^eJth feature-based relevance feedback and semantic-based relevance 
feedback. 
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20. A method as recited in claim 18, wherein the annotating jd hidden 
from the user. 

21. A method as recited in claim 18, wherein the annotating comprises: 
in an event that a particular multimedia object is deemed relevant by the 

user and not yet annotated with the keyword, adding the ^yword to the particular 
multimedia object; and 

in an event that the particular multimedia obfect is deemed relevant by the 
user and is already annotated with the keyword, strengthening an association 
between the keyword and the particular multimedia object. 

22. A method as recited in claim 1 8, wherein the annotating comprises: 
in an event that a particular multimedia object is deemed irrelevant by the 

user and is already annotated witlVihe keyword, weakening an association between 
the keyword and the particular rnultimedia object. 

23. A method as recited in claim 18, wherein the annotating comprises: 
in an event that/a particular multimedia object is deemed irrelevant by the 

user and is already Annotated with the keyword, removing the keyword from the 
particular multim^ia object. 



24. /a computer readable medium having computer-executable 
instruction^ that, when executed on a processor, perform the method as recited in 
claim 18., 
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25. A method com arising: 

maintaining associati^s between keywords and multimedia objects, the 
associations being weightep to indicate how relevant the keywords are to the 
multimedia objects; and 

retrieving a set of oie or more multimedia objects for presentation to a user; 

monitoring feedba(/k from the user as to which of the multimedia objects 
are relevant; and 

adjusting the weights of the associations based on the user's feedback. 



26. A method! as recited in claim 25, wherein the retrieving comprises 
using content-based information retrieval to retrieve the multimedia objects. 

27. A method as recited in claim 25, wherein the retrieving comprises 
using both content-ba^sed information retrieval and semantic-based information 
retrieval to retrieve the multimedia objects. 



28. A metriod as recited in claim 25, wherein the monitoring comprises 
capturing both feature-based relevance feedback and semantic-based relevance 
feedback. 



29. A me 
increasing the we 
multimedia objects 



hod as recited in claim 25, wherein the adjusting comprises 
ghts of the associations between the keywords and the 
that are deemed relevant by the user. 



7 



MStt J 5465 LI G:\MSi -O\605us\MSl-60SUS.M0Ldoc 



1 

2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 
13 
14 
15 
16 
17 
18 
19 
20 
21 
22 
23 
24 
25 



30. A method as recited in claim 25, wherein me adjusting comprises 

( j 

decreasing the weights of the associations between/ the keywords and the 
multimedia objects that are deemed irrelevant by the user. 

31. A computer readable medium / having computer-executable 
instmctions that, when executed on a processor/perform the method as recited in 
claim 25. 

32. A system comprising: 

an information retrieval unit to r^frieve multimedia objects from a database 
based on a search query; 

a relevance feedback unit to^ capture a user's feedback as to whether the 
multimedia objects are relevant toythe search query; and 

an annotation unit to apotate, with a keyword, the multimedia objects 
based on the user's feedback, 

33. A system as/recited in claim 32, wherein the search query comprises 
a keyword-based searchyquery having one or more keywords. 



34. A system as recited in claim 32, wherein the search query comprises 
a content-based search query having one or more content features. 



35. H system as recited in claim 32, wherein the information retrieval 
unit employs both content-based information retrieval and semantic-based 
information retrieval . 
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36. A system as recited in claim 32, wherein ttye information retrieval 
unit comprises: 

a query handler to handle both keyword-based gjUeries having one or more 
search keywords and content-based queries having o^e or more content features of 
a multimedia object; and 

a feature and semantic matcher to identify at least one of (1) first 
multimedia objects having keywords that match the search keywords from a 
keyword-based query, and (2) second multiiiiedia objects having content features 
similar to the content features of a content^based query. 

/ 

37. A system as recited in daim 32, wherein the relevance feedback unit 
employs both feature-based releyance feedback and semantic-based relevance 
feedback. 

38. A system as re^fited in claim 32, wherein: 

the search query emprises a keyword-based search query having at least 
one keyword; and 

in an event that^ a particular multimedia object is deemed relevant by the 
user and is not yet/annotated with the keyword, the annotation unit adds the 
keyword to the particular multimedia object. 



39. A/ system as recited in claim 32, wherein: 
the se^ch query comprises a keyword-based search query having at least 
one keywora; and 
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in an event that a particular multimedia object is deemed relevanj/oy the 
user and is already annotated with the keyword, the annotation unit steengthens an 
association between the keyword and the particular multimedia obj^t. 

40. A system as recited in claim 32, wherein: 

the search query comprises a keyword-based spdrch query having at least 
one keyword; and 

in an event that a particular multimedia ,<5bject is deemed irrelevant by the 
user and is already annotated with the keyword, weakening an association between 
the keyword and the particular multimedia object. 

41. A system as recited v/d^im 32, wherein: 

the search query compp^es a keyword-based search query having at least 
one keyword; and 

in an event that a/^articular multimedia object is deemed irrelevant by the 
user and is already a^otated with the keyword, removing the keyword from the 
particular multimedia object. 



42. An image retrieval system as recited in claim 32, wherein the 
relevance fe6dback.unit comprises a feedback analyzer to train the system based on 
the user's/feedback. 
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